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(S) High efficiency data compressed image encoding. 

@ Image data is encoded into sync blocks of 
fixed length and maximum data volume by divi- 
ding a vertical interval of image data samples, 
such as a field or frame of samples, into a 
plurality of blocks (Gi. G^,.,.) and deriving the 
orthogonal transform of the image data sam- 
ples in each block (G,, Gj,...). A block of conver- 
sion coefficients that are produced by the 
orthogonal transfonm are quantized with the 
same quantizing step to produce weighted, 
quantized converston coefficients having a 
weighting factor that is a function of the visual 
activity of that block. Then, the weighted, quan- 
tized conversion coefficients of a block are 
variable length coded and the variable length 
codes of a predetenmined number of blocks are 
combined into a sync block of fixed length. 
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BACKGROUND OF THE INVENTION 

This invention relates to image encoding and. more particularly, to high efficiency image encoding using 
an orthogonal transform, such as the discrete cosine transfonm. to produce a sync block of encoded image data. 

5 wherein the sync block is of a fixed length and exhibits maximum data volume. 

Various encoding techniques have been developed for compressing image information, such as video in- 
formation, for the purpose of transmitting image data or recording image data, such as by magnetic recording. 
Among these compression techniques are predictive coding, transfomi coding and vector quantization. 

In transform coding. tvw> perpendicular axes are used to transform image data samples, and the amount 

fo of data needed to represent the original image information, refen-ed to as "data volume", is reduced by relying 
on uncorrelated data. For orthogonal transfonn coding, the basic vectors are, of course, perpendicular to each 
other and the mean power of the conversion coefficients that are produced by orthogonal transform is substan- 
tially equal to the mean signal pow«rof the image data that is presented prior to transform. 

The conversion coefncients produced by orthogonal transfbon are known to have a DC component and 

15 several AC components, and the low frequency AC components generally exhibit a higher power concentration 
than the high frequency AC components. This pem^its the higher frequency components of the conversion coef- 
ficients to be ignored, thus reducing the data volume needed to represent the original image, without serious 
degradation in the Image that is reproduced from these conversion coefficients. Examples of orthogonal trans- 
fonn techniques include the Hadamard transfomi, Kahrunen-Loeve transform, slant transform, discrete sine 

20 transform and discrete cosine transfbmfi. The use of discrete cosine transfonn has become quite advantageous, 
and one example of the use thereof is described in U.S. Patent 5.006,931 . assigned to the assignee of the pres- 
ent invention. 

In discrete cosine transformation (sometimes referred to as OCT), an image, or more properly samples rep- 
resenting an image, is divided into several image blocks with each block consisting of n samples arrayed .n 

25 the horizontal direction and n samples arrayed in the vertical direction. That is. each block is formed of a spatial 
array consisting of n x n samples. The image data samples in each image block are processed by an orthogonal 
transform using the cosine function. The development of fast processing algorithms implemented on a single 
chip LSI circuit has enabled real time discrete cosine transfomnation of image data; and it now is not uncommon 
for OCT to be used for the transmission and/or recording of image data. Indeed, the discrete cosine transform 

30 yields an encoding efficiency that is practically equal to that of the Kahrunen-Loeve transfbmi which, in theory, 
is most favorable. The power concentration of the lower frequency components of the conversion coefficients 
produced by the discrete cosine transform is practically the same as that of the Kahrunen-Loeve transform 
which as is known, directly affects coding efficiency. By encoding only those components of the conversion 
coefficients having concentrated power, the amount of infbmiation, or data {i.e. the data volume) which need 

35 be transmitted or recorded for accurate representation of the original image is significantly reduced. 

As an example of discrete cosine transformation, let it be assumed that an 8x8 btock of image data samples 
is represented as follows: 
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in which each number in this block represents the magnitude or signal level of the image data sample. When 
the discrete cosine transfonn of the 8 x 8 block of image data samples is derived, conversion coefficients C^j (i 
represents row number and j represents column number) are produced as follows: 
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in which the number representing each conversion coefficient represents the relative power of that conversion 
coefficient. The conversion coefficient Coo "s referred to as the DC component and represents the mean lumi- 
nance value of the image block. It is seen that the electric power of the DC component is significantly higher 
15 than that of the other components which are known as AC components. As i increases, the frequency of the 
AC components in the vertical dw-ection increases and as j increases, the frequency of the AC components in 
the horizontal direction increases. As both i and j increase, the frequency of the AC components in the diagonal 
direction increases. 

The DC component of the conversion coefficients exhibits the largest value and. thus, contains the most 
20 infonmation. If the DC component is quantized with a large quantizing step, that is. if it is subjected to coarse 
quantization, block distortions are produced which appear as noise that is visually detected most readily in the 
video picture ultimately reproduced from the conversion coefficients, thereby deteriorating the quality of that 
picture. Consequently, to minimize such visual noise, the DC component of the conversion coefficients, namely 
Coo. is quantized with a small quantizing step and is represented by a larger number of bits, such as eight or 
25 more bits. A lesser number of bits may be used to represent the higher frequency AC components of the con- 
version coefficients C,| (where i, j ^ 0) because higher frequency AC components represent changes in the video 
infomnation of the n x n block and the human eye does not readily detect detail in a rapkJIy changing image. 
Consequently, an observer will not sense a loss of detail In that portion of an image which changes from point 
to point Therefore, it is not necessary to represent the higher frequency AC components of the conversion coef- 
30 ficients with a large number of bits. This means that a larger quantizing step can be used to quantize the higher 
frequency AC components of the conversion coefficients. An example of quantizing the conversion coefficients 
set out above is as follows: 
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in which the quantizing Is analogous to "rounding ofT the conversion coefTfcients. 

45 In a practical transmission or recording scheme, the quantized conversion coefficients are encoded by va- 

riable length coding, such as Huffman coding or run-length coding which provides further data compression. 
For proper transmission or recording, additional signals, such as synchronizing signals, parity codes, and the 
like, are added to the variable length coded conversion coefficients. 

In digital recording, such as a digital video tape recorder (DVTR), the amount of data which is recorded to 

50 represent a vertical interval, such as a field interval or a frame interval, preferably is of a fixed length. That is. 
although the data representing a particular block of image data samples may be variable, the total amount of 
data used to represent a predetermined number of those blocks is fixed. If a predetermined number of image 
blocks is included in a sync block, then although the amount of data (or data volume) of one image block may 
be less than that of another, the data volume of all sync blocks is substantially constant. Since the data volume 

55 of a particular image block is detemnined by the conversion coefficients produced for that block, the conversion 
coefficients of some image blocks included in a sync block may be quantized with a higher quantizing step than 
the conversion coefficients of other image blocks. Of course, when a larger quantizing step is used, less data 
is produced; and as mentioned above, it is not uncommon to quantize the higher frequency AC components 
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of the conversion coefficients with larger quantizing steps than are used to quantize the lower frequency AC 
components Accordingly, if several different quantizers are connected in common, each exhibiting a different 
quantizing step, one quantizer may be used to quantize the conversion coefficients of one image Wock and 
another may be used to quantize the conversion coefficients of another image block. As a result, the overall 
data volume of the sync block can be optimized without exceeding the data volume capacity, or preset length 
of the sync block. However, when different quantizers are selected for different image blocks, the identity of 
the quantizer which is used for a particular block must be transmitted or recorded. This identifying data does 
not represent useful image information and. thus, tends to increase the "overhead" in a sync block. This is an 
undesirable byproduct of selecting different quantizers in order to optimize data volume. 

To avoid the aforementioned "overhead", it is preferred to use the same quantizer having the same quan- 
tizing step for all of the n x n image blocks which are included in a sync block. It is expected that some image 
blocks will contain more changes in the image therein than others. Thus, the higher frequency AC components 
of the conversion coefficients in some image blocks wSI have a higher power concentration than in other image 
blocks. If these changes in the image data are refenred to as the "visual activity" of the image block, then those 
image blocks having a higher visual activity will have a smaller concentration of lower frequency AC compo- 
nents of the converston coefficients. Since, as mentioned above, the detail in that portion of an image containing 
rapid changes, that is. exhibiting a high visual activity, is not readily perceived, an image block having high ac- 
tivity can be quantized with a larger quantizing step without producing notable picture degradation. However, 
if an image block in the same sync block has low visual activity, that is. if the image block represents a monotonic 
picture pattern with a small dynamic range, the conversion coefficients are concentrated in the lower frequency 
AC components. If these conversion coefficients are quantized with a large quantizing step, that is. if they are 
subjected to the same coarse quantization that can be used fbrthe high activity image blocks in that sync block, 
block distortions resulUng in perceptible deterioration in the reproduced image is produced. Thus, although it 
is desirable to select a quantizing step that is uniform for all of the image blocks included in a sync block, if this 
quantizing step is too large, that portion of the image which does not contain high visual activity, that is. those 
image blocks which do not contain many changes, will be quantized into a number of bits that is not sufficient 
to represent that portion of the image properly. On the other hand, if a quantizer Is selected with a relatively 
small quantizing step, then an image block which contains high visual activity, that is. a large number of changes 
therein, will be represented by an unnecessarily large number of bits which is inefficient and which may exceed 
the overall limit on the data volume in the sync block. 

OBJECTS OF THE INVENTION 

Therefore, it is an object of the present invention to provide an encoding technique which overcomes the 

aforenoted drawbacks. -.»,i»,,r.e 

Another object of this invention is to encode the conversion coefficients produced by the orthogonal trans- 
fonn of an image block to maximize the data volume while quantizing image blocks coarsely if they exhibit high 
visual activity and finely if they exhibit low visual activity. . u 

A further object of this inventfon is to provide an encoding technique of the aforementioned type in which 
a predetermined number of image blocks are included in a sync block, and the same quantizing step is used 
to quantize all of the com/ersion coefficients in all of the image blocks which constitute the sync block while, 
nevertheless, providing coarse quantization for those image blocks that exhibit high visual activity and fine 
quantization for those image blocks which exhibit low visual activity. 

An addittonal object of this invention is to provide an encoding technique of the aforementioned type in 
which block distortions, quantizatton distortions and deterioration in the quality of the video picture ultimately 
reproduced from the encoded converskjn coefficients are minimized. 

Yet another object of this invention is to provide an encoding technique of the aforementioned type in which 
thequantized conversion coefficients are weighted inversely with respect to thevlsualactivityoftheimageblo^^ 

from which those conversion coefficients are derived, thereby permitting all of the image blocks in a sync block 

to be quantized with the same quantizing step. ^,,.„»f,„m 

Various other objects, advantages and features of the present invention will become readdy apparent from 
the ensuing detailed description, and the novel features will be particulariy pointed out in the appended claims. 

SUMMARY OF THE INVENTION 

In accordance with this invention, image data is encoded into sync blocks of fixed length and maximum 
data volume.Avertk»l interval of image data samples, such asafieldorframeofimagedata samples. isd|v^^ 

into a plurality of image btocks. The orthogonal transfonn of the image data samples in each image block is 
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derived to produce conversion coefficients having a DC component and plural AC components for each block. 
The conversion coefficients of an image block are quantized with the same quantizing step to produce weighted, 
quantized conversion coefficients having a weighting factor that is a function of detected visual activity of that 
block. The weighted, quantized conversion coefficients are encoded by variable length coding, and the resultant 

5 variable length coded data for each of a predetermined number of image blocks are combined into a sync block 
whose data volume does not exceed the fixed capacity of the sync block. 

As a feature of this invention, the conversion coefficients of an image block are quantized by generating 
a weighting coefficient as a function of the detected visual activity of the image block, multiplying the conversion 
coefficients of that image block by the weighting coefficient and then quantizing the weighted conversion coef- 

10 ficients with a selected one of plural different quantizing steps. As one aspect, the weighted conversion coef- 
ficients are quantized with different quantizing steps concurrently, as by plural common-connected quantizers. 
The outputs of the respective quantizers are encoded by respective variable length coders, and a sync block 
of coded data from each coder is stored. The one sync block which exhibits the highest data volume and does 
not exceed the capacity of the storage device in which it is stored is sensed, and the quantizing step which 

IS produces that sync block is selected. For example, storage overflow from a storage device is sensed, and the 
next larger quantizing step that is used to produce a sync block that does not result in storage overflow is se- 
lected. 

Alternatively, the proper quantizing step is selected by using plural common-connected quantizers, each 
with a different quantizing step, to quantize the weighted conversion coefficients. The data volume that would 
20 be attained if a given quantizing step is used is predicted: and if the predicted data volume is within a preset 
limit, the quantizer having that quantizing step is selected. 

As another feature of this invention, the visual activity of each image block is detected as a function of the 
AC components of the conversion coefficients of that block. In the preferred embodiment, the absolute values 
of the AC components* preferably the higher frequency AC components, are summed to provide an indication 
25 of visual activity. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The following detailed description, given by way of example, and not intended to limit the invention solely 
30 to the embodiments described herein, will best be understood in conjunctk>n with the accompanying drawings 
in which: 

FIG. lis a block diagram of one embodiment of an image encoder in accordance with the present invention; 
FIG. 2 is a block diagram of a digital video recorder in which the present invention finds ready application; 
FIG. 3 is a block diagram of a digital video reproducer for recovering video signals that had been encoded 
35 in accordance with the present invention; 

FIG. 4 is helpful in understanding the types of different quantizing steps that may be used when carrying 
out the present invention; 

FIG. 5 is a schematic representation of a sync block of image data in accordance with the present invention; 
FIGS. 6A-6C.are schematic representations of different Image blocks which are useful in explaining the 
w advantages derived firom the technique used with the present invention to calculate visual activity; 

FIG. 7 is a.schematic representation of how visual activity in an image block is calculated from a block of 
conversion coefficients; and 

FIG. 8 is a block diagram of another embodiment of encoder apparatus in accordance with the present in- 
vention, 

45 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

The present invention finds ready application in efficient data compression encoding of image information, 
such as video signals. The encoded image data signals may be transmitted, such as in television broadcasting 
so or video information relaying, or the encoded image data signals may be recorded, such as by a digital video 
tape recorder (OVTR). For convenience, the use of the present invention in a digital video tape recorder is de- 
scribed. 

Referring to FIG. 2. there is illustrated, in block diagram form, the primary components of a digital video 
tape recorder which incorporates the present invention for recording digitized color video signals. This recording 
55 apparatus is seen to be comprised of an analog-to-digital (A/D) converter 1 1 . a block forming circuit 1 2. an or- 
thogonal transform circuit 13. a quantizer 14. an encoder 15, a parity circuit 17 and a channel encoder 20. A/D 
converter 11 is adapted to sample luminance component Y and chrominance components U and V of an analog 
video signal. The luminance and chrominance components are supplied to input terminals 2 to which A/O con- 
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verter 11 is connected. The A/D converter operates in conventional manner to produce a vertical interval of 
pixels that represent the luminance component Y and the chrominance components U and V. The vertical in- 
terval may comprise a field of pixels or a frame of pixels. Moreover, the A/D converter may operate in a manner 
similar to line sequential sampling to produce a vertical interval of luminance image data samples as well as 
5 a vertical interval of chrominance image data samples. These image data samples are provided at output ter- 
minals 4 of the A/D converter and supplied therefrom to block fonriing circuit 12. 

The block forming circuit functions to divide a vertical interval of image data samples produced by A/D con- 
verter 11 into smaller image blocks. These image blocks are identified herein as image blocks and each 
image block is comprised of a spatial array of n x n image data samples. It will be appreciated, then, that the 
10 number of image blocks which constitute a vertical interval is a function of the number of pixels, or image data 
samples included in that vertical interval, divided by the number of image data samples which are included in 
an image block. Thus, if H represents the number of image blocks included in a vertical interval, and if N rep- 
resents the number of image data samples included in that vertical Interval, then H = N/n2. In aforementioned 
U.S. Patent 5.006.931 . it is suggested that a field interval is comprised of 2700 blocks, the number of effective 
15 lines (i.e. line intervals which contain useful video infonnation) in a field is 240 and the number of pixels in a 
line is 720. If each image block is comprised of a spatial array of 8 x 8 image data samples, then H = (720 x 
240)/(8 X 8) = 2700. Thus, if the image blocks in a vertical interval are represented as Gh, then h varies from 0 
to H, and in the aforementioned example, H = 2700. 

Orthogonal transform circuit 1 3 is coupled to block fonming circuit 1 2 and is adapted to derive the orthogonal 
20 transform of each image block supplied from the block fonning circuit to produce conversion coefficients. In 
the prefen-ed embodiment, the orthogonal transfomi is a discrete cosine transfonfn. although other orthogonal 
transfomns known to those of ordinary skill in the art may be used. The theory and operation of the discrete 
cosine transform is known; and as mentioned above, the conversion coefficients produced by deriving the dis- 
crete cosine transform of an image block contain a DC component and several AC components. If each con- 
25 version coefficient is represented as Cq (wherein [varies from 0 to n-1 and j varies from 0 to n-1 ). Coo represents 
the DC component and all other conversion coefficients are AC components. Conversion coefficients having 
higher values of i are higher frequency AC components in the vertical direction; and conversion coefficients 
having higher values of j are higher frequency AC components in the horizontal direction. 

Orthogonal transform circuit 13 is coupled to quantizing circuit 14 to supply the conversion coefficients pro- 
30 duced by deriving the orthogonal Iransfonn for each image block to the quantizing circuit The quantizing circuit 
is adapted to quantize the conversion coefficients; and as will be described, a particular quantizing step is used 
to quantize the conversion coefficients in a predetermined number of image blocks so as to avoid changing 
the quantizing step from block to block. If a sync block is comprised of this predetermined number of image 
blocks, then a given quantizing step is used to quantize the conversion coefficients produced in a sync block. 
35 As will also be described, the quantizing step is selected to provide a maximum amount of data (refenred to as 
data volume) in the sync block, provided that this quantity of data does not exceed the maximum amount of 
data that can be contained within the sync block. It will be appreciated that by using the smallest quantizing 
step that does not result in exceeding the data volume capacity of a sync block, the aforementioned problems 
of block distortion, quantizing distortion, mosquito noise and the like, all of which result in a degradation of video 
40 picture quality, are avoided, or at least minimized. 

Quantizing circuit 1 4 Is coupled to encoding circuit 1 5 to supply the quantized conversion coefficients there- 
to. The encoding circuit is adapted to encode each quantized conversion coefficient as a variable length code, 
thereby producing variable length coded conversion coefficients VLQj. Examples of suitable variable length 
codes include Huffman coding, run-length coding, and the like. It is appreciated that variable length coding pro- 
45 vides further data compression in the encoded digital signal, thus enhancing the overall encoding efficiency. 

Encoder 15 is coupled to parity circuit 17 which is adapted to generate parity bits as a function of the data 
included in the encoded sync block supplied thereto from the encoder. The parity circuit is conventional and 
adds the generated parity bits to the encoded data. The output of parity circuit 1 7 is coupled to channel encoder 
20 by way of a selector switch 18. a sync and ID insertion circuit 18 and a parallel-to-serial {PIS) converter 1 9. 
50 all connected in cascade. Selector switch 1 6 is adapted to couple the parity circuit or an external coupling ter- 
minal 6 to the sync and ID insertion circuit 18. As will be described below, the external terminal is coupled to 
the sync and ID insertion circuit during a dubbing operation, whereby an encoded video signal that is reproduced 
from another record medium is coupled directly to sync and ID insertion circuit 18 without first being decoded, 
dequantized, reconverted to analog form and then digitized, re-quantized and re-encoded. 
55 The sync and ID Insertion circuit serves to add identifying bits to the sync block for the purpose of identifying 

that sync block. If the sync block contains encoded conversion coefficients derived from, for example, image 
blocks Gi. G2. ...Gx. identification bits which serve to identify the sync block thus identify these image blocks; 
and these ID bits are generated and added to the sync block data by sync and ID insertion circuit 18. In addition. 
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a synchronizing signal nomially provided as a header for a sync block is generated by the sync and ID Insertion 
circuit. 

The output of sync and ID insertion circuit 18 is coupled to a terminal 5 to which P/S converter 19 is con- 
nected. It will be appreciated that the sync block is coupled to terminal 5 as successive multi-bit characters. 
5 such as 8-bit words. These eight bits (or any other number of parallel bits which may be used) are serialized 
by P/S converter 19 and supplied to channel encoder 20; The channel encoder processes the serialized sync 
block data to a fomn suitable for recording on a magnetic medium, such as on magnetic tape 1 by a write head 
21. Such processing may include scrambling and/or modulation, such as NR2I modulation. 

Turning now to FIG. 1, there is illustrated, in greater detail, a preferred embodiment of quantizer 14 and 

10 encoder 15, in accordance with one embodiment of the present invention. Quantizer 14 is comprised of an ac- 
tivity calculator 51 , a weighting coefficient generator 52. a multiplier 53 and a set of common-connected quan- 
tizing circuits Qi, Q2, -Qm. The quantizing circuits may be identified as Q,„. where m = 1 to M. Activity calculator 
51 is coupled to orthogonal transfomi circuit 13 and is adapted to calculate a representation of the magnitude 
of the visual activity included in an image block. As mentioned above, changes in an Image block constitutes 

15 visual activity, and an image block which contains more changes, and particularly rapid changes, exhibits a 
higher visual activity than an image block containing relatively infrequent or slowly varying changes. Accord- 
ingly, activity calculator 51 is connected to receive successive Image blocks of conversion coefficients Cjj from 
orthogonal transfonm circuit 13 and, as will be described, calculates the level of activity A^ of block Gh as a func- 
tion of the higher frequency AC components of the conversion coefficients in that image block. 

20 Weighting coefficient generator 52 is coupled to activity calculator 51 and is adapted to generate a weight- 

ing coefHcient Kh for each image block that is inversely related to the calculated visual activity A^,. Thus, as 
the calculated level of visual activity Increases, the value of the weighting coefficient K decreases. Conversely, 
as the calculated level of the visual activity decreases, the value of the weighting coefficient increases. The 
weighting coefficient Kh generated by the weighting coefficient generator is coupled to multiplier 53 which also 

25 is connected to orthogonal transform circuit 13. The multiplier serves to multiply each conversion coefficient 
C,j produced by the orthogonal transfonm circuit for an image block G^ by the weighting coefficient Kh for that 
image block. The resultant weighted converston coefficients KC|| produced by multiplier 53 are supplied in com- 
mon to quantizing circuits Qt, Q2> ...Qm- 

Each quantizing circuit operates to quantize the weighted conversion coefficient supplied thereto by a re- 

30 spective quantizing step. For example, quantizing circuit Qi may exhibit the smallest quantizing step and quan- 
tizing circuit Qm may exhibit the largest quantizing step. If q represents the quantizing step, then the quantizing 
step of quantizing circuit Qi may be q, the quantizing step of quantizing circuit Q2 may be 2q, the quantizing 
step of quantizing circuit of Q3 may be 4q, and so on. As another example, quantizing circuit Q3 may quantize 
the weighted conversion coefficient into an 8-bit signal, quantizing circuit Q2 may quantize the weighted con- 

35 version coefficient into a 4-bit signal, quantizing circuit Q3 may quantize the weighted conversion coefficient 
into a 2-bit signal, and so on. It is. of course, recognized that more infomnation. or bits, are used to represent 
a quantized weighted conversion coefficient when a smaller quantizing step is used than when a larger quan- 
tizing step is used. Stated otherwise, fine quantization is attained by the quantizing circuit exhibiting a small 
quantizing step and coarse quantization is attained by the quantizing circuit exhibiting a larger quantizing step. 

40 As will be described blow, an image block exhibiting relatively low visual activity is subjected to fine quantization 
and an image block containing relatively high visual activity is subjected to coarse quantization. Although coarse 
quantization typically results in more pronounced quantization errors, such errors are not visually perceived by 
the human eye if the image itseff contains a significant amount of visual activity. Therefore, since a highly active 
image block is quantized coarsely, such quantization errors are not readily detected. 

45 It is recognized that each quantizing circuit Qi ...Qm thus produces quantized data for a given Image block 

Gh with a data volume that differs from the quantized data produced by the other quantizing circuits. In the ex- 
ample discussed above, quantizing circuit Qi produces quantized data with greater data volume than quantizing 
circuit Qm. ■ • 

An example of the different quantizing steps exhibited by the respective quantizing circuits is demonstrated 
50 by FIG. 4. Here, an image block of conversion coefficients Cf, is represented as block 80. This block of conver- 
sion coefficients, is divided into different areas, 81. 82 and 83. and each area is associated with a respective 
quantizing step. The conversion coefficients which are disposed in each respective area are quantized in ac- 
cordance with the quantizing step assigned to that area. For example, quantizing circuit Q, quantizes the con- 
version coefficients disposed within area 81 with the quantizing step q. the conversion coefficients disposed 
55 within area 82 also with the quantizing step q and the conversion coefficients disposed within area 82 likewise 
with the quantizing step q. Quantizing circuit Q2 quantizes the conversion coefficients disposed within areas 
81 and 82 with the quantizing step q and the conversion coefficients disposed within area 83 with the quantizing 
step 2q. Quantizing circuit Q3 quantizes the conversion coefficients disposed within area 81 with the quantizing 
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step q and the conversion coefficients disposed within areas 82 and 83 with the quantizing step 2q. Quantizing 
circuit GU quantizes the conversion coefficients disposed within areas 81. 82 and 83 with the quantizing step 
2q. It will be appreciated, without further explanation, that the remaining quantizing circuits quantize the con- 
version coefficients disposed in the different areas 81. 82 and 83 with similar, differing quantizing steps. Thus. 
5 for a given image block Gh. the different quantizing circuits quantize the conversion coefficients with different 
quantizing steps, as aforesaid, to produce quantized data with different data volumes. 

It is appreciated that the conversion coefficients quantized by quantizer 14 are derived from image data 
samples representing color video signals. If the color video signals are presented in. for example, line sequential 
format, the conversion coefficients derived from the luminance component are separate from the conversion 
10 coefficients derived from the chrominance component Weighting coefficient generator 52 is controlled in re- 
sponse to luminance and chrominance switching signals supplied thereto by. for example, terminal 59. so as 
to generate a weighting coefficient for the conversion coefficients derived from the luminance component Y, a 
weighting coefficient derived from the chrominance component U and a weighting coefficient derived from the 
chrominance component V. Thus, the weighting coefficient generator is controlled, or switched, to provide a 
15 proper weighting coefficient for the Y, U and V components, respectively. 

Encoder 15 is comprised of plural variable length encoding circuits CODi. COD2. ...CODm. wherein a par- 
ticular encoding circuit CODm is coupled to a corresponding quantizing circuit Q^. Each encoding circuit CODa, 
is adapted to encode the quantized, weighted conversion coefficients KC^ by variable length coding to produce 
encoded data VLQi- Since the quantized data KC,j is produced by using different quantizing steps, it is expected 
20 that each encoding circuit produces encoded data VLC,j having a xJata volume which differs from that of the 
encoded data produced by the other encoding circuits. Thus, and consistent with the example discussed above, 
it is expected that the data volume of the encoded data produced by encoding circuit COD, is greater than the 
data volume of the encoded data produced by encoding circuit COD2, and so on. 

Each encoding circuit 000^ is coupled to a buffer memory BUFm which is adapted to store the encoded 
25 data VLC|| of an entire sync block. Each buffer memory exhibits a predetenmined storage capacity, and in the 
preferred embodiment the storage capacity of all of the buffer memories BUF,„ is the same. However, since 
the encoded data VLC^ produced by the different encoding circuits COD1 ...CODm exhibit different data vol- 
umes, it is not unusual for the data volume of the encoded data supplied to a buffer memory to exceed the stor- 
age capacity of that buffer memory. Thus, when a sync block of encoded data is written into a buffer memory. 
30 that buffer memory may overflow. A control circuit 55 is coupled to all of the buffer memories BUFi ...BUFm 
and is adapted to determine which of the buffer memories has its storage capacity exceeded by the sync block 
of encoded data supplied thereto and which of the buffer memories has a storage capacity that is not exceeded. 
Stated othen^vise. control circuit 55 detects the overflow of the respective buffer memories. 

Control circuit 55 produces an output Indication m which identifies the buffer memory whose storage ca- 
35 pacity has not been exceeded and which stores a sync block of encoded data of greatest data volume. For 
example, and consistent with the example discussed above, if none of the buffer memories overilows. since 
buffer memory BUFi stores a sync block of encoded data that is produced with the smallest quantizing step, 
the sync block of encoded data stored In buffer memory BUF, has the largest data volume and the output in- 
dication produced by the control circuit Is indicative thereof (m = 1). However, if the data volume of the sync 
40 block of encoded data supplied to buffer memory BUFi exceeds the storage capacity thereof, buffer memory 
BUFi overflows and, if buffer memory BUF2 does not overflow, control circuit 55 produces an output indication 
that buffer memory BUF2 stores the sync block of largest data volume (m = 2). Thus, depending upon which 
buffer memories overflow and which do not. the output indication produced by control circuit 55 indicates the 
particular buffer memory in which is stored a sync block of encoded data having the highest data volume. 
45 A selector 54 is coupled to each of buffer memories BUF, ...BUFm and includes a select control input cou- 

pled to control circuit 55 to receive the output indication produced by the control circuit. The selector, which 
may be thought of as a selector switch or multiplexer, selects the buffer memory indicated by the output indi- 
cation supplied thereto; and the sync block of encoded data stored in that buffer memory is read out through 
the selector as encoded data VLC. Thus, the same sync block is processed in parallel with different quantizing 
50 steps by quantizing circuits Qi ...Qm. and the processed sync block having the highest data volume that does 
not exceed the predetermined storage capacity of the buffer memory is selected. It is recognized that the se- 
lected sync block thus contains encoded data VLCy that is quantized with the minimum quantization step such 
that the data volume of that sync block is a maximum but does not exceed a predetermined data quantity. The 
sync block is comprised of a fixed number of image blocks, and by assuring that the data volume of the selected 
55 sync block is a maximum but does exceed a predetermined volume, the conversion coefficients of the selected 
sync block will be most finely quantized. 

The selected sync block of encoded data VLC is coupled by selector 54 to a multiplexer 58. This multiplexer 
also is coupled to control circuit 55 to receive the output indication m which serves to identify the particular 
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quantizing channel (formed of quantizing circuit Q^, encoding circuit COD,p and buffer memory BUF^) that Is 
selected. The multiplexer also is coupled to activity calculator 51 to receive the calculated visual activity level 
Ah of each image block Gh included in this sync block. Also, the multiplexer is coupled to a parity generator 56 
to receive the parity bits generated for the sync block: and the multiplexer is further coupled to a sync signal 
5 generator 57 to receive synchronizing signals produced at the header of each sync block. The output of mul- 
tiplexer 58 is coupled to terminal 5 (also shown in FIG. 2) to supply the sync block to, for example, parallel-to- 
serial converter 19 of FIG. 2. Although not shown, it will also be appreciated that sync signal generator 57 sup- 
plies 10 bits to the multiplexer to identify the particular sync block that is processed by the quantizing and en- 
coding apparatus. 

10 Multiplexer 58 combines the sync signals and sync block ID bits supplied thereto by sync signal generator 

57, the output indication m supplied thereto by control circuit 55. the visual activity level A^, calculated for each 
image block by activity calculator 51 . the encoded, quantized conversion coefficients VLC for each of the image 
blocks included in the sync block and selected by selector 54 and the parity bits generated by parity generator 
56 in time division multiplexed fonnat FIG. 5 is a schematic representation of the time division multiplexed data 

15 produced by multiplexer 58. By reason of the variable length coding, the length of each image block Gh of en- 
coded data may differ from image block to image block. But since the data volume of the sync block is selected 
to be a maximum, the overall length of the sync block is fixed. As illustrated in FIG. 5. there may be some margin, 
or tolerance, between the encoded data of the last image block included in a sync block and the parity bits be- 
cause the data volume of each sync block, although a maximum, nevertheless may differ from one sync block 

20 to the next However, this margin is minimal and does not detract from the efficiency of the encoding technique 
described herein. 

Since the operation of the apparatus illustrated in FIG. 1 is apparent from the foregoing discussion, further 
description thereof is provided only briefly herein. As mentioned previously, a vertical interval of image data 
samples, such as a field or frame of image data samples, is supplied by way of temiinals 4 to block fomiing 

25 circuit 12. Preferably, the image data samples of the luminance component Y are processed separately from 
the image data samples of the chrominance components U and V. The block forming circuit divides the vertical 
interval of image data samples into smaller image blocks Gh and each image block is comprised of a spatial 
array of n x n image data samples. Orthogonal transform circuit 13, which preferably is a discrete cosine trans- 
form circuit, derives conversion coefficients Cy from each image block. It is appreciated that a predetermined 

30 number of image blocks constitute a sync block. 

The conversion coefficients C|j of an Image block are supplied to activity calculator 51 which calculates the 
visual activity level A^ of that image block Gh. This calculation is determined as a function of the higher frequency 
AC components of the conversion coefficients in block Gh. The visual activity may be calculated by detecting 
the maximum value of the higherfrequency AC components, by detecting deviations among the high frequency 

35 AC components or, preferably, by summing the absolute values of the higher frequency AC components. The 
selection of fine or coarse quantization of ttie image block of conversion coefficients Is determined by the cal- 
culated visual activity. Fine quantization is selected when a low level of visual activity is calculated and coarse 
quantization Is selected when a high level of visual activity is calculated. It should be appreciated that coarse 
or fine quantization is a function of the weighting coefficient by which the conversion coefficients of the image 

40 block are multiplied. A higher weighting coefficient is equivalent to fine quantization. 

The weighting coefficient is inversely related to the calculated visual activity level such that the conversion 
coefficients are more highly weighted for lower visual activity. Thus, the conversion coefficients are quantized 
more finely for lower visual activity, even though the same quantizing circuit Qm is used for all of the image 
blocks Gh in a sync block. 

45 By using the higher frequency AC components of the conversion coefficients to calculate visual activity, 

errors in visual activity calculations that otherwise may appear in certain monotonic images are avoided. This 
is best appreciated by referring to FIGS. 6A-6C. Image block 90 in FIG. 6A represents an image having a dark 
subarea 91 and a light subarea 92 which define a vertical boundary therebetween. The AC components of the 
conversion coefficients that are derived from image block 90 exhibit higher magnitudes in the horizontal direc- 

50 tion. If, however, the Image block appears as image block 90' In FIG. 6B. formed of a dark subarea 93 and a 
light subarea.94 to define a horizontal boundary therebetween, the AC components of the conversion coeffi- 
cients exhibit higher magnitudes in the vertical direction. If the AC components of the conversion coefficients 
derived from Image block 90 or from image block 90' are simply summed, or if the maximum AC component is 
detected to calculate visual activity, the visual activity calculated for Image blocks 90 and 90' will be relatively 

55 high even though these image blocks are generally monotonic. If the conversion coefficients are quantized as 
a function of the thus calculated activity level, the conversion coefficients will be subject to coarse quantization 
even though the image blocks contain relatively low visual activity, thereby producing what is known as mos- 
quito noise in the video picture which ultimately is reproduced. 
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The foregoing error in calculating visual activity is avoided by summing the absolute value of the higher 
frequency AC components of the conversion coefficients. This is demonstrated in FIG. 7 in which a block 100 
of conversion coefficients is represented. Subarea 101 identifies that area of the block of conversion coefficients 
which contain the higher frequency AC components. If the conversion coefficients are derived from image block 
5 90 in FIG. 6A or from image block 90' in FIG. 6B. it is recognized that the AC components in subarea 101 are 
of relatively low magnitude. Therefore, by summing the absolute values of the AC components in subarea 101. 
the calculated visual activity of image blocks 90 and 90' is relatively low. Hence, the conversion coefficients 
are finely quantized, which is a proper quantization for a monotonic pattern, thus avoiding or at least minimizing 
mosquito noise. 

10 If image block 90" shown in FIG. 6C is presented for orthogonal transfonm processing and activity calcu- 
lation, the oblique boundary defined by the dark and light subareas 95 and 96 produces conversion coefficients 
whose higher frequency components are seen to Increase in the horizontal and vertical directions. Hence, the 
activity level of Image block 90" will be calculated as being relatively high, resulting In coarse quantization of 
the conversion coefficients. While this reduces the data and, thus, the detail In the encoded data that represents 

15 image block 90", quantization noise will not be easily perceived because of the rapid changes in the Image 
infonmation as shown by the obliquely extending components. 

Returning to FIG. 1, the activity level A^ calculated by activity calculator 51 for image block G^ is supplied 
to weighting coefficient generator 52 which generates the weighting coefficient Kh for this image block. The 
weighting coefficient is used to weight, or multiply, all of the conversion coefficients Cij derived from this Image 

20 block. The weighted conversion coefficients are supplied In common to the different quantizing circuits Qi ...Qm 
whereat the conversion coefficients are quantized concun-entJy with different quantizing steps, thereby produc- 
ing differently quantized data for the same Image block. Encoding circuits CODi ...CODm encode the differently 
quantized data produced for the image block Gh by variable length coding, and the variable length coded data 
thus produced by each encoding circuit is stored in a respective one of buffer memories BUFi ...BUFm- 

25 The foregoing process is repeated for each image block included in a sync block, thus accumulating in each 
buffer memory the same sync block, but the sync block stored in a respective buffer memory is quantized with 
a different quantizing step. Thus, the storage capacity of some of the buffer memories may be exceeded by 
the sync block stored therein, whereas the data volume of sync blocks stored In other buffer memories may 
be less than the storage capacity thereof. It is recognized that when the storage capacity of a buffer memory 

30 Is exceeded, the buffer memory supplies an overflow signal to control circuit 55. The control circuit supplies to 
selector 54 an output indication of the buffer memory that has not been overflowed and, moreover, that stores 
a sync block produced from encoded conversion coefficients that had been quantized with the lowest quantizing 
step which did not result in overflow. For example, if buffer memory BUF, overflows but buffer memory BUF2 
does not. the output indication supplied to selector 54 by control circuit 55 Is indicative of buffer memory BUF2. 

35 Similarly, if buffer memories BUFi and BUF2 both overflow but buffer memory BUF3 does not, then the output 
indication supplied to the selector is indicative of buffer memory BUF3. The sync block stored In the buffer menrv 
ory indicated by the output indication supplied to selector 54 by control circuit 55 Is read out from that buffer 
memory as encoded data VLC and is supplied to multiplexer 58 whereat the aforementioned sync signals, sync 
block ID bits, output indication and visual activity level all are time division multiplexed therewith, resulting In 

40 the sync block shown schematically in FIG. 5. Thus, it is seen that the sync block of encoded data VLC is com- 
prised of weighted converston coefficients derived from a predetenmined number of image blocks Gh, and the 
weighted conversion coefficients of all of these image blocks included in the sync block are quantized with the 
same quantizing step. That Is, in the embodiment shown In FIG. 1. different quantizing steps are not used to 
quantize the conversion coefficients in different image blocks included In the same sync block. 

45 Sync blocks that are produced by the encoding apparatus shown In FIG. 1 and recorded by the recording 

arrangement Illustrated in FIG. 2 may be reproduced from the record medium, such as magnetic tape 1 , by the 
reproducing apparatus illustrated In FIG. 3. As shown, this reproducing apparatus Includes a channel decoder 
32, a sync detector 34, an eiror corrector 36. a variable length decoder 37, a dequantizer 38, an inverse discrete 
cosine transform circuit 39, an Inverse block fonnfier 40. an error con-ection circuit 41 and a digital-to-analog 

50 converter (DAC) 42. Channel decoder 32 is coupled to a playback head 31 to receive the modulated signals 
that had been recorded on magnetic tape 1 by write head 21 of FIG. 2. Accordingly, the channel decoder is 
adapted to demodulate and/or descramble the reproduced signals to recover digital infomnatlon therefrom. As 
an example. If the recorded signals had been modulated with NRZI modulation, the channel decoder includes 
an NRZI demodulator. 

55 Channel decoder 32 is coupled to sync detector 34 by way of a serial-to-parallel (S/P) converter 33. It is 

recalled that multi-bit characters had been serialized by PIS converter 19 prior to recording: and S/P converter 
33 functions to return the serialized multi-bit characters into parallel form. These parallel-bit characters are sup- 
plied to sync detector 34 whereat the sync signals Included in the sync block shown In FIG. 5 are detected. 
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This assures synchronous detection and recovery of the reproduced sync blocks. 

The sync detector is adapted to reproduce the encoded data VLC,) which is coupled to error corrector 36 
by way of time base conrector 35. The time base corrector is a conventional device that is used for eliminating 
jitter and other time base errors that may be produced during the reproduction of the encoded data from mag- 
5 netic tape 1. Once jitter has been conrected, or removed, error corrector 36 functions to correct errors that may 
be present in the recovered sync block by using the parity bits included therein. If, however, the parity bits cannot 
be used to conrect an error, for example, if the error is too large, an error flag is set to identify the uncon-ectable 
error. 

Then, after en-or correction, the sync block is coupled to variable length decoder 37 which decodes the 
10 variable length encoded, quantized conversion coefficients. In particular, decoder 37 operates to complement 
the variable length coding that had been carried out by one of the encoding circuits CODi ...CODm, shown in 
FIG. 1 . Accordingly, decoder 37 recovers the quantized conversion coefficients; and these quantized conver- 
sion coefficients are coupled to dequantlzer 38 which operates to complement the selected quantizing circuit 
Qi ...Qm that had been used for recording. The sync block that had been recorded and reproduced from mag- 
15 netic tape 1 includes output Indication m therein, such as shown in FIG. 5. and this output indication is used 
to identify the quantizing step that had been selected to quantize the conversion coefficients. It is appreciated 
that the complementary dequantizer is selected by this indication m to effect a proper dequantization. 

Once dequantized, the conversion coefficients are coupled to inverse discrete cosine transform circuit 39 
which operates in a manner Inverse to that of discrete cosine transform circuit 13. Hence, the conversion coef- 
20 ficients Cjj are used by the inverse discrete cosine transform circuit 39 to recover the n x n array of image data 
samples included in each of the image blocks that comprise the sync block. The resultant, recovered image 
blocks of image data samples are re-fomied into a vertical interval, such as a field or frame, by inverse block 
former 40. 

The recovered vertical inten/al of image data samples are coupled to error correction circuit 41 wherein 

25 those en-ors that could not have been corrected by error corrector 36 are corrected here by more sophisticated, 
known techniques, such as by interpolation. Such error corrections are achieved by using the enx)r flags that 
had been set by error corrector 36. 

The error-corrected re-fonmed vertical interval of image data samples are coupled to DAC 42 whereat they 
are converted firom digital form to analog form and supplied as analog signals to output terminals 3. Since the 

30 luminance and chrominance components are processed and recorded separately, as discussed above, the re- 
producing apparatus illustrated in FIG. 3 recovers separately the luminance and chrominance components Y 
and U, V to be further coupled to other circuitry (not shown). 

It will be appreciated that sync detector 34 is adapted to recover the output indication m and the visual ac- 
tivity level Ah (shown in FIG. 5) from the recorded sync block. Output indication m. which represents the quan- 

35 tizing step used during the recording operatten. is supplied to decoder 37 and to dequantizer 38 to select the 
proper dequantizing circuit which exhibits the corresponding quantizing step. Similariy. the recovered visual 
activity level A^ is supplied from sync detector 34 to dequantizer 38 to select the proper inverse weighting coef- 
ficient with which the decoded, dequantized conversion coefficients are multiplied. This restores the weighted 
conversion coefficients to their original, unweighted values. It is seen that since the visual activity level A^ de- 

40 tennines the weighting coefficient Kh. it is a simple matter to select the inverse weighting coefficient 1 1K^ from 
this same visual activity level. 

The VTR-recording apparatus shown in FIG. 2 and the VTR playback apparatus shown in FIG. 3 are par- 
ticulariy adapted for dubbing digital video signals reproduced from one magnetic tape onto another. In a typical 
digital VTR. digital video signals which are reproduced from a magnetic tape generally are subjected to digital 

45 processing, which includes error correction, decoding, dequantizing. inverse transformation and digital-to-ana- 
log recording, to recover analog signals which then are digitized, transfomied. requantized and encoded before 
re-recording. As will be appreciated, the steps of quantizing/dequantizing. transfonnning/inverse-transfomnlng 
and A/D/A conversion not only introduce errors into the original information but amplify such errors. For exam- 
ple, A/D conversion inherently produces a small digitizing error, and this small en-or is compounded by D/A 

50 conversion which results in an analog signal whose level is expected to differ somewhat from the original analog 
signal level. Likewise, the converston coefficients produced by deriving the orthogonal transfonm of digital video 
samples, although a very dose approximation of the original video information nevertheless is not a precise, 
exact representation. Therefore, the signal processing operations which are carried out to play back and then 
re-record (or dub) digital video signals with high efficiency coding introduces errors into the re-recorded digital 

55 video signals. When the higher frequency components of the conversion coefficients C„ are subjected to coarse 
quantization, as is desired in order to limit the bandwidth needed to represent the digital video signals, proc- 
essing errors are accumulated when a dubbing operation is executed, thereby degrading the quality of the video 
picture eventually reproduced from the video signals. 



11 



EPO 517 256 A2 



This drawback is overcome by providing dubbing output taps, or terminals, in the VTR recording apparatus 
as well as in the VTR playback apparatus which, when interconnected, effectively bypasses those circuits which 
inherently introduce enrors into the processed digital signals and permit the direct re-recording of digital signals 
without subjecting those signals to D/A/D conversion. These output taps are shown as output terminals 7 (FIG. 
5 3) and 6 (FIG. 2) provided in the reproducing and recording apparatus, respectively. Output terminal 7 is coupled 
to the output of error corrector 36 to provide an error connected, time-base-conrected parallel digital signal rep- 
resenting the quantized, weighted conversion coefficients in the predetermined number of image blocks which 
constitute a sync block. That is, the sync block illustrated in FIG. 5 is provided at output terminal 7. and this 
output terminal is connected to external terminal 6 in FIG. 2. When in the dubbing mode, selector switch 16 of 
10 the recording apparatus shown in FIG. 2 is coupled to external terminal 6 to supply the sync block reproduced 
by the reproducing apparatus of FIG. 3 to parallel-to-serial converter 19 and channel encoder 20 for re-record- 
ing on magnetic tape 1 by write head 21. Thus, the digital video data reproduced by read head 31 is dubbed 
without being decoded, dequantized, inversely transformed, D/A converted, AID converted, orthogonally trans- 
fonmed, requantized and re-encoded, as is the case with prior art dubbing anrangements. Consequently, digital 
75 video infonmation which had been compressed with desirably high efficiency is not subjected to inherent proc- 
essing enrors when that compressed digital video information is dubbed onto another record medium. 

Another embodiment of quantizer 14 and encoder 15 of the digital video recording apparatus in accordance 
with another embodiment of the present inventkjn is illustrated in FIG. 8. Here, the quantizer is comprised of 
a buffer memory 64. a weighting circuit 65. a selectable quantizer 66 and a data volume estimator 70. Buffer 
20 memory 64 is coupled to an orthogonal transform circuit 63 to receive the conversion coefficients Cij produced 
thereby. As before, the orthogonal transfonnn circuit may be a discrete cosine transform circuit and functions 
to derive the conversion coefficients in an image block of image data samples produced by block forming circuit 
62. These image data samples are digital samples generated by an AID converter 61 from analog video signals 
coupled thereto by way of an input terminal 68. It will be appreciated, therefore, that A/D converter 61. block 
25 forming circuit 62 and orthogonal transform circuit 63 may be substantially similar to aforedescribed A/D con- 
verter 11. block forming circuit 12 and orthogonal transform circuit 13, shown in FIG. 2. 

Buffer memory 64 is adapted to store a sync block of conversion coefficients which subsequently is read 
out to weighting circuit 65. Thus, the buffer memory exhibits a storage capacity sufficient to store the conversion 
coefficients Cq derived from image blocks 6,, G^, ...Gh. where h is the predetemnined number of image blocks 
30 which constitute a sync block. Weighting circuit 65 may comprise a multiplier similar to aforedescribed multiplier 
53 (FIG. 1) and is adapted to multiply, or weight the conversion coefficients read out firom buffer memory 64 
with a weighting coefficient Kh. wherein Kh is the weighting coefficient that is determined for the particular image 
block then being read from the buffer memory. The weighting coefficient is produced by a classifying circuit 
72 included in data volume estimator 70. as will be described. 
35 The weighted converston coefficients KC,j produced by weighting circuit 65 are coupled to selectable quan- 
tizer 66 which is comprised of a plurality of quantizing circuits Qi. Q2, ...Qm that may be similar to aforedescribed 
quantizing circuits Qi...Qm of FIG. 1. The quantizing circuits included in selectable quantizer 66 thus exhibit 
different quantizing steps; and for convenience, it is assumed that quantizing circuit Qi exhibits the smallest 
quantizing step to produce fine quantization and quantizing circuit Qm exhibits the largest quantizing step to 
40 produce coarse quantization. These quantizing circuits are coupled to a selector 66a which is controlled by a 
selector control signal produced by a data volume predictor 75 included in data volume estimator 70. As will 
be described below, the data volume predictor is adapted to determine the smallest quantizing step that should 
be used to quantize the conversion coefficients in a sync block which will produce a maximum data volume 
that does not exceed the data volume capacity of the sync block. Selector 66a selects this quantizing circuit 
45 and couples the output thereof to an encoder 67. Hence, encoder 67 is supplied with a sync block of quantized, 
weighted conversion coefficients exhibiting the smallest quantizing step that can be accommodated by the sync 
block. 

Encoder 67 preferably is a variable length encoder similar to encoding circuits COD, ...CODm shown in FIG. 
1. The encoder functions to code the quantized conversion coefficients supplied thereto by selector 66a with 
50 variable length coding, such as Huffmann coding, run-length coding, and the like. The variable length coded 
data VLC produced by encoder 67 Is coupled to an output terminal 69 which, for example, may supply the en- 
coded data to parity circuit 17 of FIG. 2. 

As mentioned above, data volume estimator 70 is adapted to estimate, or predict, the smallest quantizing 
step that can be used to quantize the weighted conversion coefficients so as to produce a sync block of max- 
55 imum data volume which, nevertheless, does not exceed a preset data volume limit That is. the maximum data 
size, or limit, of the sync block shown in FIG. 5 cannot be exceeded. The data volume estimator includes an 
activity detector 71. a classifying circuit 72, a variable quantizer 73, a multiplier 74 and data volume predictor 
75. Activity detector 71 may be similar to aforedescribed activity calculator 51 and is adapted to calculate the 
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level of activity of an image block as a function of the higher frequency AC components of the conversion 
coefficients Qj included in that image block, as supplied to the activity detector by orthogonal transfomn circuit 
63. Classifying circuit 72 is coupled to activity detector 71 and is adapted to produce the weighting coefficient 
Kh as a function of the activity level detected for the image block then being supplied to the activity detector. 

5 This weighting coefficient Kh is coupled to weighting circuit 65. as mentioned above, and also to multiplier 74. 
As one example, classifying circuit 72 is adapted to classify the activity level of an image block Gt, into high 
and low activity levels, depending upon the value of the activity level A^ produced by activity detector 71 . If the 
activity level An exceeds a predetermined threshold THE, the classifying circuit classifies the image block as 
a high activity block; and if the detected activity level Ah is less than this threshold level THE, classifying circuit 

10 72 classifies the image block Gh as a low activity block. For example, the weighting coefficient Kh produced by 
classifying circuit 72 may be Kt, - 1/2 if the activity level Ah is greater than the threshold level (Ah > THE) and 
Kh = 1 if the activity level is less than the threshold level (Ah < THE). 

Variable quantizer 73 is coupled to orthogonal transform circuit 63 to receive successive image blocks of 
conversion coefficients Cq. The variable quantizer also is supplied with a control signal fed back thereto from 

IS data volume predictor 75 which Is used to selectively change the quantizing level of the variable quantizer. For 
example, if the data volume predictor estimates the data volume of a sync block to be greater than the preset 
limit, or data capacity, of that sync block when a particular quantizing step is used by variable quantizer 73. 
the control signal fed back to the variable quantizer increases the quantizing step which, in turn, decreases the 
estimated data volume of the sync block. If the data volume now estimated by data volume predictor 75 for this 

20 increased quantizing step still exceeds the preset limit, the control Signal fed back to variable quantizer 73 caus- 
es a further increase in the quantizing step. As a result, a particular quantizing step is selected which is the 
smallest quantizing step that can be used to quantize the conversion coefficients without exceeding the data 
volume capacity of a sync block. 

Variable quantizer 73 is coupled to data volume predictor 75 by multiplier 74, This multiplier may be similar 

25 to multiplier 65 and is adapted to multiply the quantized conversion coefficients produced by the variable quan- 
tizer by the weighting coefficient Kh produced by classifying circuit 72. Thus, data volume predictor 75 is sup- 
plied with weighted, quantized conversion coefficients so as to provide a good approximation of the data volume 
resulting from the quantized, weighted conversion coefficients. 

Those of ordinary skill in the art recognize that the image infonmatlon in adjacent image blocks formed of 

30 8x8 pixel arrays does not change significantiy. Although the image information in adjacent image blocks may 
not be identical, a large change in such image information is not expected and is highly unlikely. Therefore, 
from the aspect of statistical analysis, a reasonably good estimation of the amount of data needed to represent 
a predetermined number of image blocks, such as the number of image blocks which constitute a sync block, 
can be made on the basis of the amount of data that is needed to represent one of those image blocks. This 

35 is the function of data volume predictor 75. For example, variable quantizer 73 may be set initially to exhibit 
the smallest quantizing step. The conversion coefficients included in the first image block of a sync block are 
quantized with this quantizing step and then weighted in multiplier 74 by weighting coefficient K. The data vol- 
ume predictor operates to estimate the data volume of this image block based upon the quantizing step selected 
for variable quantizer 73. The estimated data volume of the sync block may be approximated by multiplying 

40 the estimated data volume of the image block by the number of image blocks included in the sync block. If the 
estimated sync block data volume exceeds the capacity thereof, data volume predictor 75 feeds back a control 
signal to variable quantizer 73 to increase the quantizing level. Accordingly, when the next Image block of con- 
version coefficients is supplied to the variable quantizer by orthogonal transform circuit 63. such conversion 
coefficients are quantized with this larger quantizing step. The data volume predictor once again estimates the 

45 data volume of this image block; and that estimated data volume is used to approximate the data volume of 
the sync block if this higher quantizing step is used. If the approximated data volume still exceeds the preset 
limit, the foregoing feedback operation is repeated. Eventually, the sync block data volume approximated by 
data volume predictor 75 does not exceed the sync block data volume capacity; and the quantizing step which 
had been selected to produce this approximation is represented by the selector control signal that is supplied 

50 from the data volume predictor to selector 66a. thereby selecting the quantizing circuit having a correspond- 
ing quantizing'step. 

As each image block of conversion coefficients is supplied in succession to variable quantizer 73, that im- 
age block also is supplied to and stored in buffer memory 64. It will be appreciated that, even in the worst case, 
data volume predictor 75 and variable quantizer 73 cooperate to select the smallest quantizing step to produce 
55 maximum, pennissible data volume (i.e. a data volume that does not exceed the preset limit) before an entire 
sync block of conversion coefficients is stored in the buffer memory. After a sync block has been stored, the 
conversion coefficients are read out from buffer memory 64 to weighting circuit 65 whereat they are weighted 
with Uie weighting coefficien K produced by classifying circuit 72. The weighted conversion coefficients KCij 
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are supplied in common to quantizing circuits Qi ...Qm; selector 66a responds to the selector control signal 
produced by data volume predictor 75 to select the quantizing circuit having the smallest quantizing step which 
has been estimated to result in a sync block data volume that does not exceed the sync block data capacity. 
That is, the fixed length of the sync block (as shown in FIG. 5) is not exceeded. 

5 The selected quantizer is used to quantize the weighted conversion coefficients for an entire sync block, 

even though the weighting coefficient may vary from block to block; and the quantized, weighted conversion 
coefficients thus produced are encoded in variable length code by encoder 67. The variable length coded con- 
version coefficients VLC are supplied to, for example, multiplexer 58 (FIG. 1 ) by way of output tenminal 69. Sim- 
ilarly, the selector control signal produced by data volume predictor 75 also is supplied to the multiplexer by 

10 way of output terminal 76. This selector control signal is used in the same way as the output indication m pro- 
duced by control circuit 55 of FIG. 1. That is. the selector control signal is used during a playback operation to 
select the proper dequantizer which is complementary to the quantizing circuit that had been selected by se- 
lector 66a during recording. 

Thus, it is seen that the embodiment shown in FIG. 8 achieves the same desirable objectives as the enrv 

15 bodiment of FIG. 1 , wherein the same quantizing step is used to quantize conversion coefficients for an entire 
sync block while, nevertheless, providing fine quantization for those image blocks having a relatively low level 
of visual activity and coarse quantization for those innage blocks having a relatively high level of visual activity. 
By weighting the conversion coefficients with a weighting factor that is inversely related to the detected visual 
activity of an image block, fine quantization can be obtained for an image block having low visual activity by 

20 weighting the conversion coefficients with a higher weighting factor than is used for an image block having high 
visual activity. It will, of course, be recognized that when a larger quantizing step is used to quantize a conver- 
sion coefficient having a larger weighting factor, the result is equivalent to fine quantization. Conversely, when 
that same quantizing step is used to quantize a conversion coefficient having a smaller weighting factor, the 
result is equivalent to coarse quantization. 

25 In the embodiments shown In FIGS. 1 and 8, activity calculator 51 and activity detector 71 operate to cal- 
culate or detect visual activity of an image block Gh as a function of the higher frequency AC components 
of the conversion coefficients C(|. In one embodiment, the activity level is detemnined from the sum of the,ab- 
soiute values of such conversion coefficients. For example, the sum of the absolute values of the conversion 
coefficients C,,. where i = 6 or 7 and j = 0-7. represent the high frequency components in the perpendicular 

30 direction of an image. This provides a reasonably accurate indication of visual activity, in another embodiment, 
such as shown in FIG. 7, the absolute values of conversion coefficients Cq. wherein i = 4-7 and j = 4-7. may 
be summed. 

Whilethe present Invention has been particularly shown and described with reference to preferred embodh 
ments, 11 will be readily appreciated by those of ordinary skill in the art that various changes and modifications 

35 may be made without departing from the spirit and scope of the invention. For example, weighted conversion 
coefficients can be obtained without the use of separate weighting circuits, or multipliers, 53 and 65 (shown in 
FIGS. 1 and 8). As shown in FIG. 1. multiplier 53 may be bypassed, as represented by the one-dot-chain line, 
such that the conversion coefficients Cq produced by orthogonal transfonm circuit 13 are coupled directly to 
quantizing circuits Qi ...Qm- In this alternative, the weighting coefficient K produced by weighting coefficient 

40 generator 52 is coupled to control circuit 55, as represented by the one-dot-chain line, and is used by the control 
circuit to produce the output indication m by which selector 54 selects the quantizer-coder-buffer channel de- 
termined by the weighting coefficient Thus, depending upon the detected visual activity A^ of an image block, 
that image block is quantized with a particular quantizing step which has the same effect as quantizing a weight- 
ed conversion coefficient. That is, in the alternative embodiment shown by the one-dot-chain lines in FIG. 1. 

45 the conversion coefficients Cq of an image block are quantized with different quantizing steps concun*ently by 
common-connected quantizing circuits Q, ...Qm» and one of these quantizing steps is selected by selector 54 
under the control of control circuit 55 as a function of the weighting coefficient K supplied to the control circuit 
by weighting coefficient generator 52. Thus, even in this modified embodiment, weighted, quantized conversion 
coefficients having a weighting factor that is a function of the detected visual activity of the image block are 

50 produced, variable length coded and supplied by selector 54 to multiplexer 58. 

In the embodiments described herein, orthogonal transfonn circuits 13 and 63 may be implemented by a 
digital signal processor (DSP) or by a suitably programmed microprocessor. Orthogonal transforms and the 
implementations thereof are known to those of ordinary skill in the art Preferably, the orthogonal transfomn is 
a discrete cosine transform; but as noted above, other known types of orthogonal transfonmation may be used, 

55 if desired. 

It is intended that the appended claims be interpreted as covering the particular embodiments specifically 
described herein, those alternatives which have been mentioned above, and all equivalents thereto. 



14 



EP 0 517 256 A2 



Claims 

1. Image encoding means for encoding image data samples comprising: 

block-fonming means (12: 62) for forming plural blocks of image data samples, each block compris- 
5 ing a spatial an-ay of n x n image data samples; 

orthogonal transform means (13; 63) for deriving the orthogonal transform of the image data sam- 
ples of each block to produce conversion coefficients having a DC and plural AC components for each 
block; 

activity detection means (51; 71) responsive to AC components of the conversion coefficients of a 
10 block to detect visual activity of said block; 

weighting means (52; 72) for weighting the conversion coefficients of said block as a function of 
the detected activity; and 

quantizing means (Qt, Qz Qm) ^or quantizing the weighted conversk)n coefficients. 

15 2. The apparatus of claim 1. wherein said activity detection means (51; 71) is operable to produce an Indi- 
cation of visual activity of a block as a function of absolute values of said AC components of the conversion 
coefficients in the horizontal and vertical directions. 

3. The apparatus of claim 1 or 2« wherein said orthogonal transform means (1 3; 63) comprises discrete cosine 
2^ transfonm means for deriving a discrete cosine transform of said image data samples for each block. 

4. The apparatus of anyone of claims 1 to 3, wherein said quantizing means (Q^. Q2 Qm) comprise plural 

quantizers (Qi, Q2 Qm)* each having a different respective quantization step. 

5. The apparatus of claim 4, wherein said quantizing means includes selector means (54, 66a) for selecting 

one of saki quantizers (Q,, Q2 Qm) to quantize the weighted conversion coefficients of a predetermined 

number of blocks, whereby the same quantization step is used for each of said predetenmined number of 
blocks notwithstanding changes in the visual activity in said blocks. 

6. The apparatus of claim 4 further comprising variable length encoding means (COD1, C0D2,...»C0Dm; 67) 
^ for encoding quantized, weighted conversion coefficients from each of said quantizers (Qi, Q2 Qm) and 

adapted to produce different variable length output codes, and selector means (54) for selecting the va- 
riable length output code produced by said encoding means (COD1, C0D2,....C0Dm) having minimum 
quantization width and maximum data volume. 

35 7. The apparatus of claim 6, wherein said encoding means (COO1, COD2 CODm) comprises plural variable 

length encoders (CODt, C0D2,...C0Dm)* each coupled to a respective quantizer (Qi, Q2,...,Qm) to produce 
a respective variable length output code, and wherein said selector means (54) is further operable to select 
the variable length output code whose maximum data volume does not exceed a predetermined length. 

40 8. The apparatus of claim 6, wherein sard selector means (66a) is coupled to said quantizers (Q,, Q2,...,Qm) 
for coupling the quantized, weighted conversion coefficients produced by one of said quantizers (Q,, 
Q2.....Qm) to said encoding means (67), and further including selector control means (75) for controlling 
said selector means (66a) to select said one quantizer (Qi, Q2,...,Qm) as a function of predicted data vol- 
ume of the variable length output code. 

45 

9. The apparatus of anyone of claims 6 to 8. further comprising control means (75) for controlling said selector 
means (54; 66a) as a function of detected visual activity. 

10. A method of encoding image data into sync blocks of fixed length and maximum data volume, comprising 
^ the steps of: 

dividing a vertical interval of image data samples into a plurality of image blocks (Gi, 62....): 
deriving the orthogonal transform of the image data samples In each image block (Gi. G2,...) to pro- 
duce conversion coefficients having a DC component and plural AC components for each image block 
(Gi,G2,...): 

detecting the visual activity of each image block (Gi, G2,...) 

quantizing the conversion coefficients of an image block (G^, G2,...) with a common quantizing step 
to produce quantized data constituted by weighted* quantized conversion coefficients having a weighting 
factor that is a function of the detected visual activity of said image block (Gi, G2....); 
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encoding the quantized data of an image block (G,. G2....) by variable length coding to produce en- 
coded data; and 

combining the encoded data of a predetermined number of image blocks (Gi, G2,...) into a sync 
block of encoded data. 

11. The method of claim 10, wherein the step of quantizing comprises generating a weighting coefficient for 
an image block (Gi, G2,...) as a function of the detected visual activity of that image block (Gi, G2....). mul- 
tiplying the conversion coefficients of an image block (Gi, Gj....) with said weighting coefficient to produce 
weighted conversion coefficients, and quantizing the weighted conversion coefficients with a selected one 
of plural different quantizing steps. 

12. The method of claim 11, wherein the step of quantizing with a selected one quantizing step comprises 
quantizing the weighted conversion coefficients with different quantizing steps concurrently to produce dif- 
ferently quantized data, variable length coding each of the differently quantized data, storing a plurality of 
sync blocks, each containing variable length coded data produced by a different quantizing step, and se- 
lecting the quantizing step which produces the sync block exhibiting the highest data volume which does 
not exceed a given storage capacity. 

13. The method of claim 12, wherein the step of selecting the quantizing step comprises sensing storage over- 
flows of respective ones of the stored sync blocks, and selecting the next larger quantizing step used to 
produce a sync block that does not result in storage overflow. 

14. The method of claim 11. wherein the step of quantizing with a selected one quantizing step comprises 
quantizing the weighted conversion coefficients with different quantizing steps concunrently to produce dif- 
ferently quantized data, predicting the data volume of each sync block formed of differently quantized data, 
and selecting the quantizing step which produces the sync block having the largest predicted data volume 
within a preset limit 

15. The method of daim 14. wherein the step of predicting the data volume of each sync block comprises: 

(a) selecting a prequantizing step of minimum value for prequantizing the conversion coefficients of an 
image block (Gi, G2....) 

(b) weighting the prequantized conversion coefficients of each image block (G,, G2....) to produce 
weighted prequantized data. 

(c) calculating an estimated data volume from the weighted prequantized data of an image block (Gi. 
G2....), 

(d) detennnining if the estimated data volume exceeds said preset limit. 

(e) increasing the value of said prequantizing step if the estimated data volume exceeds said preset 

limit, 

(f) repeating steps (a) - (d). and 

(g) sensing when the estimated data volume does not exceed said preset limit 

16. The method of claim 15, wherein the step of selecting the quantizing step comprises: 

(h) selecting a quantizing step corresponding to the value of the smallest prequantizing step which re- 
sults in an estimated data volume that does not exceed said preset limit. 

17. The method of claim 10. wherein the step of quantizing comprises generating a weighting coefficient for 
an image block (Gi G2,...) as afunction of the detected visual activity of that image block (Gi. G2,...), quan- 
tizing the conversion coefficients of an image block (Gi, G2....) with different quantizing steps concunrently. 
and selecting one of said quantizing steps as a function of said weighting coefficient 

18. The method of anyone of claims 10 to 17, wherein the visual activity of each image block (Gi, G2,...) is 
detected as a function of the AC components of the conversion coefficients of that block. 

19. The method of claim 18. wherein the step of detecting the visual activity of each image block (Gi. G2....) 
comprises summing the absolute values of higher frequency AC components of the conversion coefficients 
of that block. 

20. The method of anyone of claims 10 to 19, wherein the step of deriving the orthogonal transform comprises 
driving the discrete cosine transfomn of the image data samples in each image block (Gi, G2.. •) 
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